As the scale and scope of network-based applications and networked-based services such as cloud computing have increased, data centers may house hundreds and thousands of host devices (e.g., web servers) that need to be load balanced. Some network host devices such as load balancers generally receive incoming traffic to a network and distribute or route it to other network devices (e.g., to services deployed on multiple hosts). Conventional load balancers typically use techniques such as max connections, round robin, and/or least connections applied to data collected from the host devices to select which host device will handle a connection. In addition, load balancers may serve as proxies to the host devices they front and thus terminate connections (e.g., Transmission Control Protocol (TCP) connections) from client and send client traffic to host devices on connections established between the host devices and the load balancer. Thus, a host (e.g., a back-end device or process) and a client do not necessarily communicate over a direct connection when using load balancers.
In some instances, network devices may need to forward extra information along with the original request to the other network devices. Some mechanisms for forwarding the extra information lack robustness. For example, forwarding mechanisms that rely upon parsing the application request may fail and skip adding the metadata. In another example, forwarding mechanisms that rely upon integration efforts (e.g., code to strip out metadata from TCP payload) may not always be reliable.
Further, while health probes (e.g., TCP or HTTP pings) may be used to identify hosts which are not able to process any requests, some probes (e.g., a TCP ping) may not be reliable for detecting some or all application level issues, while other probes (e.g., application or service level probes such as HTTP pings) may not be comprehensive enough to cover some potential failure paths. In other words, a health pin may only indicate that a host is available, not information about the host's load. Therefore, solely depending on these probes to establish the health status of hosts may error prone and unreliable.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.